Array-camera motion picture device, and methods to produce new visual and aural effects

ABSTRACT

A system and method for creating special effects comprising capturing an image of an object or objects in motion from each of a plurality of cameras aligned in an array, assembling the images from each camera into a series of images in a manner providing a simulation of motion from a perspective along the array with respect to the object or objects within the images when a series of images is displayed in sequence, wherein certain object or objects appear substantially stationary in time during the simulation of motion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 09/727,034 filed on Nov. 30, 2000, now U.S. Pat.No. 7,843,497, issued on Nov. 30, 2010, which is a continuation of U.S.patent application Ser. No. 08/598,158, filed on Feb. 7, 1996 (nowabandoned), which is a continuation of U.S. patent application Ser. No.08/251,398, filed on May 31, 1994 (now abandoned). The entire disclosureof each of the above applications is incorporated herein by reference.

FIELD

The present disclosure relates to the production of new kinds of visualand aural effects for display in motion picture, video and audio venues.

BACKGROUND & SUMMARY

Traditionally, a stream of images, recorded by a single motion picturecamera, or video camera, is displayed upon a screen to produce anillusion of motion. If a rotational effect of a subject were desired,one would circumnavigate the subject with a motion picture camera anddisplay this series to simulate the rotational effect. But, how wouldone freeze a subject in one position, say a diver, entering the pool,with water-splashing up all around, and create a rotational motionpicture effect about the frozen diver? To freeze the diver in oneinstant, traditionally one would need to circumnavigate the diver inalmost no time (approximately 1/2000 second or less), with a super-highframe rate motion picture camera. I believe that no suchcircumnavigational camera device exists. If one wants to freeze, thenvisually rotate an even faster subject, such as an artillery shellleaving the muzzle of a large military gun, one would need tocircumnavigate the speeding shell in 1/500,000 second or less. What ifone wanted to walk into a room full of fluttering butterflies, have thebutterflies appear to freeze in their current positions, and be able tocinematically record and display a motion picture simulated “walk”through this room of frozen butterflies? You can do these things, andmore, using arrays of cameras, pre-positioned around, or through asubject area. Then sequentially displaying the records made by the manymembers of these arrays.

Several inventors (see examples below) have suggested methods employingarcurate or circular arrays of camera devices to capture differenthorizontally displaced photographic records of a subject with the objectof facilitating the production, or reproduction of works of sculpture orof providing 3D still or motion picture representations of the subject.None has suggested, or, in my judgment, anticipated the methods andmechanisms to produce the useful and novel frozen effects describedabove and other kinds of effects described below in my specification.

EXAMPLES

Smith U.S. Pat. No. 891,013 Jun. 16, 1908 Ives U.S. Pat. No. 1,883,290Oct. 8, 1932 Ives U.S. Pat. No. 2,012,995 Sep. 3, 1935 Staehlin, et alU.S. Pat. No. 2,609,738 Sep. 9, 1952 Strauss U.S. Pat. No. 2,928,311Mar. 15, 1960 Collender U.S. Pat. No. 3,178,720 Apr. 13, 1965 Glenn U.S.Pat. No. 3,518,929 Jul. 7, 1970 Matsunaga U.S. Pat. No. 3,682,064 Aug.8, 1972 Collender U.S. Pat. No. 3,815,979 Jun. 11, 1974 Danko, Jr. et alU.S. Pat. No. 4,010,481 Mar. 1, 1977 Collender U.S. Pat. No. 4,089,597May 16, 1978 Collender U.S. Pat. No. 4,158,487 Jun. 19, 1979 Ross U.S.Pat. No. 4,199,253 Apr. 22, 1980 Morioka U.S. Pat. No. 4,239,359 Dec.16, 1980

It is one of the objects of the teachings to provide a mechanism andmethod to capture and display a motion picture-like rotational effect ofan animate subject, like looking at a revolving statue of the subject.This subject can be rotated upon any existing and ordinary motionpicture screen, or television screen to simulate the visual effect ofwalking around, and visually inspecting a statue of the subject.

Another object of the teachings is to use novel shapes and dispositionsof camera arrays in combination with new methods of assembling andpresenting these records to produce other novel effects. Camera arrayshapes, such as, but not limited to long chains of cameras, in linear,or curvilinear arrays are employed. These arrays can be operated insynchrony or non-synchrony to capture different angular visual recordsof a subject area. These different records can be sequentially displayedto create the novel visual effect of traveling linearly, orcurvilinearly along the chain, through a frozen moment of time. Theeffects will be similar to the tableaux effects in theatrical plays.Animate objects like people are frozen in time, yet one character getsto move through this moment.

A more generalized object of the teachings is to provide powerful, newvisual and/or aural perceptions of the world, employing methods in whicharrays of various receiver devices, such as, but not limited to, cameradevices, or microphones, or combinations thereof, capture differentangular records of energy emanating from a subject of interest. Whicharrays are of many and variable shape, e.g. circular, arcurate, linear,curvilinear, dome-like, or many other shapes. Which arrays are comprisedof members that can be individually manipulated, positioned, aimed, andoperated, before and during energy capture, by hand, or by remotecontrol, or remote computer control, in synchrony or non-synchrony.Recordings made by the many array members are captured, manipulated, andcombined into many and variable sequences, and presented according tomethods described below to provide said novel visual and/or auralperceptions.

Further areas of applicability will become apparent from the descriptionprovided herein. The description and specific examples in this summaryare intended for purposes of illustration only and are not intended tolimit the scope of the present disclosure.

DRAWINGS

The drawings described herein are for illustrative purposes only ofselected embodiments and not all possible implementations, and are notintended to limit the scope of the present disclosure.

FIG. 1A shows 10 video camera devices, arrayed in a horizontallycircular array around a diving area. In practice, from 6 to hundreds ofcameras would be employed in this array. Long focal length lenses wouldenable distant camera placement, allowing a large number of arraymembers. 150 computer; 152 video storage (tape, disc, or other); 153alternative on-camera storage (as in High Eight); 156 pan/tilt servos;158 and 160 are electrical or fiber optic communication paths betweencomponents, operator, and audience;

FIG. 1B shows frames of a diver, ready to be rotated;

FIG. 1C shows alternative frames of a diver, ready to be rotated;

FIG. 2 shows a curvilinear array of camera devices, 20-n, in Plan andFront views. Cameras 20-103 take in a leftward view. Cameras 113-n takein a rightward view. Cameras 104-112 (series E) change view fromleftward to rightward. Cameras 104-112 (series E) rise sequentiallyabove the field of view of preceding cameras. “A” is the view seen bycamera 101. “B” is the view seen by camera 102. “C” is the view seen bycamera 114. “D” is the view seen by camera 115;

FIG. 3A shows rod-shaped ROC targets (154), imaged by array members. 162is an axis of rotation. (See FIG. 2 above for description of othernumbered items.);

FIG. 3B shows enlarged ROC target. 162 is axis of rotation;

FIG. 4 shows an array of transparent, beam-splitter like camera devices.Cameras see through one another, obviating need for rise series E inFIG. 2. The image of the butterfly seen by the furthest camera in thelinear array is dimmer than the image seen by the closest. 174 arecameras, and 170 are beam splitters;

FIG. 5 is an outline for a program which recognizes an image of atarget, and records the changes necessary to bring this target imagefile into conformity with an ideal target image;

FIG. 6 is an outline for a program which recognizes an image of atarget, and records the changes necessary to bring a camera array tobear so that it captures target images which are in conformity with anideal target image;

FIG. 7 is an outline for a program which recognizes bright points oflight projected onto a subject, identifies these points as seen byadjacent camera array members, and assigns these morph point locationsto visual images of the subject captured by this same camera array;

FIG. 8 shows an alternative embodiment of two views of a curvilineararray of camera devices according to the present invention;

FIG. 9 shows an array of beam splitters as an alternative embodiment ofthe present invention and relating to the embodiment seen in FIG. 4;

FIG. 10 shows an arrangement method to squeeze a greater amount offrames of visual data onto a length of color film;

FIG. 11 shows two views of a curvilinear array of camera devicesaccording to the present invention set up to record butterflies;

FIG. 12 illustrates the use of glass sheets in association with thecamera array of the present invention;

FIG. 13 is an alternative view of the beam splitter shown in FIG. 4; and

FIGS. 14 and 15 represent camera configurations according to the presentteachings.

Corresponding reference numerals indicate corresponding parts throughoutthe several views of the drawings.

DETAILED DESCRIPTION

Example embodiments will now be described more fully with reference tothe accompanying drawings.

In order to illustrate our methods, several new motion picture effectsare described below. Also described below are effects produced by arraysof other types of receiver, like microphones, or flow meters, whichillustrate our methods to study energy flow.

For one example, in order to analyze light energy reflecting from thesurfaces of an Olympic diver, we arrange a plurality of motion videocameras into a horizontal ring around the diver, with all cameras aimedand focused upon the same point, and all adjusted to take in the entiredive area. In FIG. 1A, we see the diver surrounded by the camera ring.Here, 10 of a multitude of cameras are shown. In practice we would usefrom 8 cameras, to hundreds of cameras. In this example, the cameras arefixedly mounted at 15 degrees to the horizontal to avoid including othercameras in the scene. Cameras are gen-locked to synchronize imagecapture on either internal tape (such as “High Eight” 8 mm) or externalmulti-track video recorder. Infra-red focus maintains sharp focus ofmoving diver. Lighting and aperture are selected to provide good subjectfocus while giving little background detail. In this instance, weoperate our cameras in synchrony, capturing frame after frame of visualdata as he dives through the air and into the water. We choose a momentin time, say that moment when the diver just begins to slice the water.We choose the frame recorded most closely in time to that instant, sayframe number 248, and have a video control computer select frame 248from each camera by frame-grab control and plant each in sequence on an“output” tape. Displayed traditionally, at 30 video frames per secondonto a video screen, this output tape produces a rotational effect; likelooking at a rotating statue of this diver, frozen at this instant oftime, with even the water droplets frozen in mid-air. This display wouldblend and enhance the more usual action tapes of the event. The effectappears on a traditional television screen, and the viewer can sit andwatch while this amazing, beautiful and useful effect is displayed. Theviewer does not have to get up out of a chair and walk about a displaymechanism to enjoy this effect.

We could then continue our analysis in this example by choosing asequence taken by one particular motion video camera in our array, anddisplaying this new sequence in reverse order from frame 248, backwardto a moment when the diver was in mid-flight; perhaps frame 220. Wemight then freeze and rotate the diver in mid-dive, at frame 220,analyze the crucial moment, then rotate back to any camera position, andallow the action to continue forward, displaying frames 220, 221, 222,etc., from this camera angle. The diver would then be released fromfreeze, and would complete his entry into the water.

In my first experiments with array cameras, an array of forty-one stillphotographic cameras was deployed into a ninety-degree circular arc.Each camera was aimed roughly at a vertical target located at the vertexof this arc, and a frame of film in each camera was exposed insynchrony. The images of an assistant as she ran through this array,with cameras on bulb, and while she is constantly illuminated bytungsten light have been joined and displayed at 24 frames per second toform a stunning new visual effect. The tunnel of light which she createdcan be rotated upon the screen through the ninety degrees, revealing newand marvelous visions of human locomotion. At one end of the rotationalarc, the tunnel of light is oriented from left to right across thescreen. At the other end of the rotational arc, she is running directlyout toward the viewer. All angles in between were recorded by the arc,and during display, these angles flash in sequence upon the screen tosimulate walking around this statue of a frozen stream of time. It'sstunning. Another display series formed during these early sessionsreveals the inventor, standing at the vertex of the array, pouring waterfrom a pitcher into a goblet. Each camera captured an image of thisaction in synchrony with the other array cameras. Exposure was made by abrief, 1/2,000 second burst of light from a studio strobe. This briefexposure froze the water droplets in mid-air as they sprayed out fromthe goblet. Upon display, the series taken from the array creates asimulation of a frozen statue, which can be rotated back and forththrough ninety degrees on screen.

The array of cameras need not be an arc. We could, for instance, arrangea plurality of cameras in a dome shape, positioning individual camerasso that many different chains of cameras can be traced out for a varietyof display series.

Or, we could form a curvilinear array and operate it to form a trackingeffect. For example, we would arrange our cameras and employ our methodsto simulate a walk through a room filled with frozen butterflies. Referto FIGS. 2 and 8 to find still cameras curvilinearly arrayed down a paththrough this room, with array members pointing first left (cameras20-103; cameras 1-n), then rising smoothly through series E whileshifting gaze to the right (cameras 104-112), then continuing throughthe room pointed rightward (cameras 113-n). The angles of view ofadjacent cameras (A, B) and (C, D) slightly overlap. This is a techniquefamiliar to animators. If the subject image were to leap too far, frameto frame, the displayed result would form a strobe-like effect. Overlapfrom image to image provides a smooth, coherent result. The brightly litroom is full of live, fluttering butterflies. All shutters are made torelease at one moment, at high shutter speed, (say 1/1000 second) tocapture a frame of visual data. We select that frame from each camera,and arrange them in sequence from camera 20 to camera n, or camera 1 ton in FIG. 8, on a storage medium such as, but not limited to motionpicture film, video tape, optical or magnetic storage disc, or RAMmemory. We then display this sequence rapidly as is done in motionpicture or television display. (Twenty-four frames per second is thetheatrical motion picture standard, and 30 frames per second is the NTSCstandard, but any rate which is rapid enough to form a coherent visualeffect may be employed.) This rapid display forms a visual simulation oftravel through this room full of living butterflies. The simulationbegins as we travel into the room, gazing slightly to the left,according to our array configuration (FIG. 2, cameras 20-103; FIG. 8,cameras 1-103). Near the middle of the room we rise as we shift gaze tothe right (E, cameras 104-113), and proceed through the room, gazingslightly rightward (cameras 113-n). Cameras rise out of the precedingcameras' view from 104 to 113.

If we display the frames from these cameras at 24 frames per second, thetrip through the room would be n/24 seconds long. The rate of travelalong a display sequence taken from a chain array of cameras could becontrolled by varying the distance between adjacent cameras, by varyingthe rate of display, or by altering the display sequence from 1, 2, 3,4, etc. to 1, 1, 2, 2, 3, 3, 4, 4, etc. The rate of travel can also bealtered by using interpolation methods to compute frames between thosecaptured by array members and including these frames in our displaysequence. (See interpolation discussion below.) Animators frequently“shoot on twos”; that is, they display each frame twice, effectivelydoubling the duration of the sequence. We could shorten the duration,(speed up the travel rate) by skipping every other frame, as in 1, 3, 5,7, etc. The animator must bear in mind that there are limits beyondwhich the human perceptual system cannot integrate displayed series intocoherent movement. A good book on animation technique will help thepractitioner of our methods to design arrays, manipulate, and displayimages so that the result conforms to the needs of the human perceptualsystem.

We need not capture or display only one image per frame, per camera.Multiple exposures can be captured on one frame of film. This one frameof multiple images can be selected from each camera, combined in simplesequence from camera to camera and displayed to form a rotational effectof this multi-image frame. Or, we can super-impose several frames andproject them in sequential “packs” projecting frames 1, 2 and 3 packedtogether as one frame, then 2, 3, 4; then 3, 4, 5; etc. to form powerfuland novel effect.

When we speak of freezing a moment for analysis, we mean, of course thatwe record a “short interval” of time for analysis. Even a millionth of asecond is a stream of time. So, short moments can be frozen andanalyzed, or longer moments can be “frozen.” For example, it isinteresting to allow a subject to move through the target zone of ahorizontally elliptical still camera array while the cameras are onbulb, thus capturing a tunnel of moving light on each frame of film;which tunnel can be rotated, or otherwise displayed visually accordingto our methods.

We would employ a computer controlled timing device to control thetiming and sequence of array member energy capture to produce noveleffects. This would be a relatively simple computer program which wouldbe capable of accepting instructions from a human operator and passingthese instructions on to array members; which instructions detail thetime at which each particular array member was to begin and endrecording. Array members could be hard-wired to this control device (asin Matsunaga U.S. Pat. No. 3,682,064), or could be controlled byinfrared or radio transmission. For instance, in a simple ring of 200still cameras, one could time the array to capture first a frame fromcameras 1 and 200 simultaneously, then 2 and 199 simultaneously, then 3and 198, and so on, to 100 and 101, with a new pair of exposures beingmade at equally spaced time intervals so that the last pair, 100;101 ismade ½ second after the first pair 1;200. If the cameras each capture abrief, “freezing” exposure of the subject, then selecting, arranging anddisplaying frames 1, 2, 3, 4, 5, 6, etc. would produce the effect ofmoving around the subject from camera position 1 to 200; and at the sametime the subject would change its position. At camera position 100 inour displayed series, the subject would reverse itself and move back toits original position, while we continue to move around the subject from101 to 200. Thus, simulating the effect one would get by properlycombining for display the film footage taken by two super-humanly fleetfooted motion picture camera operators who had raced along a circulartrack about the subject and had filmed the subject with super high framerate motion picture cameras, one operator racing clockwise, the othercounter-clockwise.

Other array timing patterns could be useful, for example, in the arrayabove, expose 1, 3, 5, 7, 9, . . . 199 at one instant, then 2, 4, 6, 8,10 . . . 200 the next. Displaying 1, 2, 3, 4, 5, simulates a rotationaleffect about the subject, while the subject moves back and forth fromits position in the odd series to its position in the even series. Thistype of series display could help us to analyze vibrationalcharacteristics of structures. It is but one example of how the timingof the array is variable to create new analytical tools.

There are difficulties to be overcome when employing my methods.Employing camera arrays, one must be certain that the images are inregistration with each other so that upon display, images do not jitterabout annoyingly, or even incoherently. This is a problem familiar totraditional animators.

To illustrate: Locate a statue of an athlete. Take a motion picturecamera, and place it on a wheeled cart which runs on an elliptical trackabout the statue. Take frame after frame of film of the statue, whilerolling around on the cart. On projection, one sees a statue of thisathlete, rotating up on the screen. But, if the track has manyimperfections, the cart will bounce up and down, pitching, rolling, andyawing drastically along. Each time the camera exposes a frame of film,the camera's orientation to the statue is displaced not only by a smoothelliptical increment about the track, but by some additional, unwantedfactors. If the amount of unwanted displacement were too great, thenupon projection, the image of the statue would jitter and jump about onthe screen so much that the human mind-could not integrate image afterimage into a coherent rotational effect.

We may encounter this tracking problem with our camera array. If thecameras, one after another, pitch, roll and yaw, from camera to camera,by an undesirable amount in orientation to each other, or in orientationto the athlete, then, upon projection we will encounter the same jitterand jump problems mentioned above. Or, in our curvilinear trackingexample above, if camera after camera were not arrayed with smoothlychanging orientation to the visual subject, and to one another, theeffect would not cohere into a simulated “walk through a frozen moment.”To avoid these problems, array members must be positioned with greataccuracy. Focus, focal length, aim, position along the array, distancebetween cameras, orientation to subject, and orientation to adjacentarray members must all be controlled so that a displayed series ofimages taken from a chain of array members forms a coherent effect. Thefollowing methods facilitate such accurate positioning of array members.

First, we choose a reference target, of which a 2D image reveals thetargets 3D orientation to the array. Please refer to the rod-shapedtargets in FIG. 3A. The target is placed so that a computer program (seeFIG. 5) can recognize the target against its background. If the targetappears sufficiently different from the background in hue or brightnessa computer program can separate the image pixels of the target from theimage pixels of the background. We place the reference target inside ourelliptical array, and record an image of the target with eachpre-oriented camera. Each camera's orientation to the target, and to oneanother can be deduced for registration purposes. Individual cameradeviation from, say, a desired vertical orientation is revealed by anon-vertical rod image created by that camera. Sphere image sizeindicates distance from the target (or, focal length adjustment). Anideal sphere size indicates proper camera distance/focal lengthadjustment. An ideal size sphere coupled with a shorter than ideal rodimage indicates an angular displacement of the camera, under or over thetarget. Different colored portions of the target indicate, in the image,which portion of the target is rotated toward or away from the camera.And so on. A computer program (See FIG. 6) would analyze thesepositional attributes and would determine the camera adjustmentsnecessary to, for instance, achieve vertical rod position, and to centerthe sphere portion of the target on the display screen, or, to changethe dimensions of the target image (mechanically zooming in or out, ordollying in or out) so that the sphere remains the same size from cameraimage to camera image. These array attributes are adjusted according tothis image analysis to maintain properly registered image size, focus,and orientation during display of the visual effect. These arrayattributes can altered by hand for each array member, or by remotecontrol means using electromechanical or mechanical rotational, panning,tilting, trucking, tracking, focusing, focal length adjusting devicesattached to each array member. These remote functions could becontrolled by computer program, which program would use the results ofthe above image analysis to establish and maintain proper image size andorientation in each camera. A series displayed from this properlyconfigured array would show the target rod rotating smoothly about anaxis drawn from target rod end to end (see FIG. 3B).

In practice, this degree of accuracy in array placement and operationwill be difficult to maintain in a camera array which is beingdynamically reconfigured to capture the action on an athletic field.Therefore, in addition to these array attribute manipulations, one couldmanipulate image records prior to recording them, or prior to displayingthem, so that remaining registration problems are alleviated.

A computer automated process to perform these functions would facilitateinstant replay effects of an athlete during competition. For example,prior to the diving events, our geometric target would be temporarilysuspended in the dive space. Each camera in our apparatus would bepre-aimed at this target, and its view recorded. A computer programwould analyze the recorded image of the target provided by each camera.If one of our cameras were aimed slightly and improperly tilted to theleft, then the computer would see a target shape slightly tipped to theright. The program would measure this deviation, and would create a filerecord of the changes necessary to rotate this camera's output tovertical. We would not necessarily need to re-orient the camera itself,only the camera's output. A record is made for each camera, and theserecords are called a “Record of Changes” or (ROC) file. When we latercall for an instant replay of diving action, our program will quickly beable to manipulate the image from each camera according to (ROC)instructions, and feed the corrected images into the stream, forming ourrotational effect.

In practice, the following series of steps would be performed to achievea dynamically manipulable array, whose capture and display attributeswould allow instant replay effects. First, a human operator or acomputer program would direct camera array, shape, member position alongthe array, member orientation to the subject, member orientation toadjacent array members, member aim, and focus and focal length asaccurately as possible, bringing the array into proper adjustment tocapture the data necessary to produce the desired display result. Theseadjustments would be effected using servo type mechanisms, and othermechanical or electromechanical devices attached to each camera member.The computer would then fine tune the array positional and opticalattributes by target and ROC method. Finally, ROC file data would beused to change recorded image attributes (prior to image storage, or atreplay); which changes would alleviate remaining registration problemsupon display.

In my early photographic experiments mentioned before, my ROC targetconsisted of 2 small, spherical objects. The first sphere secured to thefloor, the second suspended by a thread, directly above the first. Eachof the 41 cameras was aimed roughly at this target area, and successiveframes of film were exposed to capture a moment of action from all 41camera locations, all at once. To assemble a projectable series offrames, I first recorded the image of the ROC target as seen by eachcamera. Later, an analysis of this image revealed each camera'sorientation to the target. Each image was then manually alteredaccording to ROC information to form a projectable rotational effect.

The target need not be a rigid, physical one. It could be a target ofreflected light, projected into the area of activity, reflecting fromthe surfaces of the particulate matter suspended in the air. Forinstance, one would project several, narrow laser beams so that thesebeams cross one another to form three, different-colored, bright spotsin space. These pin-points in space would be bright enough to bedetected by the cameras in our array. Different colored lasers could beused. Or, one could pulse them at distinctive rates to make themdifferentiable.

If the light were invisible to human vision, either above or below thevisible spectrum, or low enough in power, or of short enough duration tobe humanly imperceptible, then these points of light could be projectedduring the athletic event. In an instant, we could re-aim to a new areaof interest, project a target into that area, and use ROC target methodsto fine tune array orientation and image output.

If the subject of interest moved to a new location, but was still inview of the array members; we could, without moving the array members,project a target to that new area, and calculate a new ROC. The computerwould apply the new ROC to manipulate each cameras output to form arotational effect about that new location.

Or, this same capability could be acquired by calculating a multitude ofROCs in advance, for a particular array orientation. The axis ofrotation associated with each ROC target would be plotted upon a map ofthe athletic area. One could then, instantly choose a particular ROCfrom the map which corresponds to the area about which we now wish torotate. For instance, multiple targets, physical or projected, might beplaced, one at a time, or as a group, along the path which a diver islikely to travel (See FIG. 3A). These targets would be imaged by thearray. If one wished to rotate about the diver just as the diver comesoff the board, one would choose the ROC from the target which was atthat location, and computer manipulate the images from our circulararray to form a rotational effect about that axis. If we wish to capturehim as he enters the water, we would choose the ROC target which residedat that location. And so on.

One might pre-establish several different camera array configurationswhich would produce acceptably registered display series. Theseconfigurations would be noted in computer memory taking note of allarray attribute adjustments, and the positions of the mechanical orelectromechanical devices which control these attributes. When the areaof visual interest changed during an athletic event, the array would bedynamically adjustable to a multitude of pre-established configurationsto enable effect capture for use in display. Mechanical means, such asdetent-like devices, might be used in place of computer memory toestablish array configuration. Several detent positions might bepre-established, into which the array attributes can be snapped to coveran event dynamically.

A dense, 3-D lattice-work of laser beams could be projected throughspace, from many different sources. This lattice could serve as amultitude of targets, and a ROC could be calculated for any axis ofrotation covered by the lattice.

Without projecting our ROC targets remotely, we could still place targetshapes into the athletic arena in such a way that they did not interferewith the athletes. (For example, humanly invisible light sources couldbe placed outside the area of activity, but still visible to our cameraarray. Perhaps off to the side, and/or above the area of activity. Evenstationary equipment on the field might serve as ROC targets; e.g. goalposts, markings on the field, court, etc.

In curvilinear, or linear arrangements of cameras, one would choose aROC target shape like a picket fence which each camera could see as itmoved through the subject area. One would aim or manipulate images sothat verticals stay vertical, and picket images are captured and/ordisplayed so that they move at the desired rate across the screen.

These methods need not be restricted to the capture and analysis ofvisible light. One might wish to analyze sound energy, or some humanlyinvisible form of electromagnetic energy. We might wish to measurefluid, flow, and employ some array of flow meters to do so.

If one wished to analyze sound from some source employing this method,our ROC target might then be 3 discreet sound sources in the targetarea, each emitting timed pulses of omnidirectional sound, and eachemitter operating at a different frequency. A plurality of microphonesmight be arrayed about this target area, and one might analyze the soundgathered by this array, at a particular instant in time. One could thencalculate the microphones' orientations to the target area by looking atthe incoming wave form received by an array member and comparing it toan ideal, and comparing it to the wave forms being gathered by its arrayneighbors. One might then use these comparisons to adjust the aim of thearray members in relation to the sound source, bringing array members tobear on the source, or causing them to diverge from the source accordingto our desired method of analysis. One might want to simulate the soundheard during a walk away from a source, or a turn away, or a flight upover, etc. One could use the ROC target method to adjust our arrayand/or its output to effect the desired result.

Or, one might wish to combine aural and visual information according toour methods. For example, if our subject were a bat (animal), one mightchoose to couple a microphone to each camera, forming a combined array.We might choose a short stream of visual information from each member ofthe array ( 1/1000 second shutter speed) to freeze, and pair each ofthese “frames” with a 1/1000 second stream or “frame” of auralinformation, then display this series according to our method. Thus,simulating a rotating statue of a bat, frozen and screeching, at thatone instant.

We could employ camera devices which are sensitive to an extendedelectromagnetic range and could slowly, during display, begin to replacethe humanly visual spectrum data with data collected above or belowvisual range. As we begin to exceed the upper boundary of the visualrange, we could signal this fact visually by adding dots, or bands, tothe display; perhaps changing the size of the markings to correspond tohigher and higher wavelength. Or, we could begin to replace the firstinvisible ultraviolet, with the lowest visible red from the spectrum,reusing the visual portion of the spectrum to represent this nexthighest sector. The first time we reused the visual portion, we couldsuperimpose tiny markings upon the display, then as we ascend farther,we could use larger markings, and so on. We could employ a similarprocedure as we descend below the human visual spectrum. We wouldreplace the first invisible infrared with the highest visible violet.Or, we might display only one visual frequency at a time as our subjectrotates, then proceed up or down the spectrum, one frequency at a time.

We could do the same sort of thing with sound, reusing the auralspectrum up and down the range of possible pressure wave frequencies. Wemight create a visual representation of the sound. Perhaps, the louderthe sound or the higher the pitch, the more we would tint the picturered. We could then “see” a representation of the sound as well as hearit.

We might similarly create a visual, and aural representation of fluidflow, by using flow meters in place of microphones in the above array.On display, we could substitute aural data to represent flow data.Higher flow could be represented as a higher pitch, so that as wevisually “walk around” the flow source, or flow subject, we could watchit and listen to it. As our visual perspective changed from camera tocamera, our aural perception of flow would change as well. We could hearthe flow. Or, higher flow could be represented as higher visual lightfrequency, as is done in some computer modeling of air flow. In ourmethod, we would present data, captured from the real world, and thenrepresented according to our methods visually and/or aurally.

In all of these instances of the collection and analysis of energy, wecould resort to computer interpolation techniques to fill the gapsbetween energy gathering device members of the array. It seems to me,however, that it will usually be desirable to fill these gaps withactual energy gathering devices, if at all practical, rather than to tryto infer the energy traveling through that space. Nature has a way ofsurprising us.

To illustrate such a surprise, consider that we might have an array of 2cameras, aimed and focused at a point 3 meters away. Our subject is anarrow tube. This tube is oriented so that if one places his eyedirectly between our cameras, and looks at the tube, he can see down itslength to a glittering diamond. But, the tube is long, and narrow, andthe diamond is far in and cannot be seen by either camera 1 or camera 2.We do not currently know how to interpolate an image of this diamond byusing information from cameras 1 and 2. This 2-camera array isimperfect. The human visual system does something analogously imperfect,taking images from an array of 2 eyes, and forming what seems to be acomplete view of the subject. It may be, however, that we are not seeingall of the visual information emanating from this front view.

Imagine, for example, an extremely narrow laser beam which could beplaced directly in front of us, so that it shone directly upon the spot,midway between our eyes. If the beam source were extremely small, and ittraveled through a vacuum so as not to be reflected or refracted bymatter, this beam could be placed at some distance from us, hitting usright between the eyes, and we would not see it. Neither eye membercould provide sufficient information for our brain to compute theexistence of this beam. If we moved our array so that this beam enteredthe optical system of an eye, suddenly, as if from nowhere, a blazingpoint of light would appear as the laser's beam blasted away at theretina. There is almost certainly visual data from the real world thatbehaves this way. Even when array members are placed as closely togetheras are human eyes, visual information slips past. We may some day beable to substitute for an array, a continuous light sensitive surfacewhich would record holo-graphically a complete record of theinterference pattern created as reference and subject beam strike thissurface. We might, then, have a perfect array.

In practice, there will be times when interpolation is desired. We maynot be able, for financial or technical reasons, to create a perfectarray for our needs; but we may be able to use interpolative methods toapproximate the function of a perfect array. For example, we might wishto set up an elliptical array of cameras about our subject. We'd like tobe able to analyze all visual data passing through this ellipse, but wecan only physically accommodate a certain number of taking devices inour array. So, we would use interpolation methods to guess at theappearance of our subject as it might appear from angles between ourarray members.

The process of image interpolation called morphing is well known, and wewould proceed according to its established principles in order tointerpolate from image to image, but we would suggest a method whichautomatically assigns morph points to our subject.

Traditionally, an animator who wishes to morph from one image to thenext, must manually assign sets of morph points which the morphingprogram uses to gradually transform image 1 into image 2. In ourexample, we wish to compute a series of images such as would be seenfrom angular positions between 2 adjacent cameras in our array. We wantto be able to display this series upon a screen so that it appears thatwe have rotated our subject, slowly from the view seen by camera 1 tothe view seen by camera 2. If our subject were a vertically orientedhuman face, and our circular camera array looks down on this face from45 degrees above an imaginary horizontal plane running through themiddle of the face, then upon elliptical rotation, the tip of the nosemust move from image 1, elliptically across our screen, and wind up inproper location at the tip of the nose in image 2. All other details ofthe face must move and match up with their counterparts in image 2.Furthermore, all of these points must move according to their locationin real space. The tip of the nose will move along a larger ellipse, andmore rapidly across the screen, than will the bridge of the nose. At thesame time, the visual data must be smoothly transformed, incrementallywith our computed series, so that hue, and brightness values change fromthose in image 1 to those in image 2.

One could manually assign morph points to several bodily features, butthe time required to assign such a large number of pairs would precludeinstant replay. An automatic method would enable instant replay, andwould assign such a dense covering of morph points, perhaps even toevery pixel of every camera image in the array, that morphing would beaccomplished with maximum accuracy.

One method to automatically assign morph points would be to pepper thesurfaces of the subject with thousands, or even millions of differentcolored pinpoints of reflected light. As many colors as are humanlydifferentiable could be used to indicate to a human morph point selectorwhich points in successive camera images are to be paired as morphpoints. Or, a computer could be programmed to recognize these differentcolor point locations (see FIG. 7). Since 24-bit color processors canrecognize over 16 million colors, we could project a dense array of 16million different colored points onto our subject's surfaces. Thecomputer would then be instructed to analyze 2 or more images betweenwhich we wished to interpolate. Points of light of the same colorappearing in the 2 or more images would be assigned as correspondingmorph points. This morph point data would be captured simultaneously orintermittently with visual data of the subject. And the morph point datacaptured by each camera would be applied to the visual data captured bythat same camera.

Simultaneous capture could occur if the light frequencies employed inthe dot array were above or below those used in the visual data. Visualdata would be used to form our effect, point data would be used toestablish morph points.

Intermittent collection could occur if we pulsed the points of light, sothat at one instant, the subject were illuminated by the multitude ofcolored points, and the next instant the subject were illuminated byfull-spectrum white light. We would collect morph point informationintermittently with visual data. The location of each morph point wouldthen be matched to a location on the visual image data. For example, amotion video camera could be designed to read out frames of data at twotimes the normal rate. One frame would be read as the subject wasilluminated by colored points, then a frame of visual information wouldbe read as the subject was illuminated by white light. If the subjectdid not move too far in the interval between morph point collection andvisual data collection, one could rely on sufficiently accuratecorrespondence between the image locations of morph point data andvisual data. If the subject were moving rapidly, the rate ofintermittent morph point projection and white light projection, andsynchronous frame collection would be increased, or, at least, theinterval between a frame of morph point data collection and a frame ofvisual data collection would be decreased so as to assure adequatecorrespondence between subject position in morph data image and visualdata image.

Sixteen million colors may not be necessary. Colors could be reused inthe array, provided that same color dots appeared sufficiently far aparton the subject surface. The computer would then look at images, and findcorresponding colors in confined image sectors. For instance, a deepblue dot in the top ¼ sector of the image area would not be matched witha deep blue dot in the lower ¼ sector of the image area.

Light reflecting from one point on the subject surface might take oncolor information in reflecting from that surface. The light reflectingfrom that point toward camera 1 might be color altered differently fromthe light reflecting from that point toward camera 2. We might overcomethis problem by designing our array of points so that each point in thearray is surrounded by points of greatly different wavelength. Thecomputer would be instructed to look for close matches in wavelengthfrom a particular image area. For example, in a horizontal circulararray of 20 cameras around a dancer. A blue dot shining off the tip ofher left elbow might appear ½ of the way up from the bottom of theimage. In the camera 2 image, this dot will also appear approximately ½of the way from the bottom. The computer would then be instructed tolook for blue dots, near to this color, on this approximate band ofhorizontal image data from the camera array. Finding a close match, theprogram will assume that the dot is the same color, and will assignmorph points accordingly to the visual data which corresponds spatiallyto this morph point.

In another method to automatically assign morph points to a subject, wewould first measure the precise geometric dimensions of our subject, andform a mathematical model of the subject's 3D surface locations. Wewould then match the model to visual data collected by our array, thusforming morph points automatically.

Methods of remote measurement have been described as in Ross, U.S. Pat.No. 4,199,253; Apr. 22, 1980. Please refer to this patent specification.Such a method could be used to remotely measure a subject. We wouldemploy a plurality of such measuring systems. The angles from whichthese Ross systems projected and collected radiant energy data would bechosen so that the subject's surfaces of interest were all well coveredwith diagnostic imagery. The more complex the shape of the subject, themore angles might be required. For instance, a dancer might move duringour analysis so that her arm blocked the diagnostic light from a Rosssystem transmitter from reaching her torso. In order to avoid suchshadows, one would have to ensure that diagnostic radiant energy wereprojected from a plurality of angles, around, and perhaps above andunder the subject.

Three dimensional measurements taken from a multitude of angles aroundthe subject would be combined by computer program to form a mathematicalmodel of the subject according to triangulation methods well known inthe computer modeling area.

We would measure in advance, the orientation of the measuring devices tothe camera array, and could then match 3D data to the 2D visual datacollected by the camera array, thus automatically forming morph pointson the surfaces of each display image between which we wished to morph.We would measure Ross system orientation to camera array by ROC targetmethod, first using the Ross systems to measure the dimensions of a ROCtarget such as seen in FIG. 3A of my specification. We would thenmeasure the difference in size and orientation of the target as seen bythe Ross systems and the camera array members. A visual representationof this process would include displaying the 3D computer model of theROC target in 2D projection on a video screen. A record would be made ofchanges in the 3D model necessary to bring its 2D projection intoconforming size, shape, and orientation with the ROC target images ofeach camera array member.

We would now know what the subject looks like from each camera position,and we would know the three spatial dimensions of the subject'ssurfaces, and we would know how these dimensions appear as seen fromeach camera position. We could then pair these data sets to form amultitude of morph points for each camera image. Each pixel, of eachimage would be accurately assigned to its proper location in 3D space.In effect, each pixel of each image would become a morph point.

The interpolation of images between camera locations would then proceedaccording to known morphing technique. To form a rotational effect, eachpixel, from each camera image, would be instructed to move across thescreen in an elliptical manner about an axis of rotation. Points on thesubject close to the axis of rotation would move on smaller ellipses,traveling shorter distances per unit time than would points on thesubject farther from the axis. Pixels, representing light reflected fromthese points, and assigned 3D coordinates, would move across the screenaccording to simple geometric rules. And, as they moved, these pixelswould slowly take on the hue and brightness values of theircorresponding morph points as seen from camera x+1.

One could form a different effect by interpolating from camera image tocamera image captured by a linear array of cameras, such as wasdescribed earlier in the “walk through the butterflies” example. Theautomatic light point mentioned above would assign morph points to theimages from adjacent cameras. Pixels of an image from camera 1 wouldmove under common morph program control in linear fashion across thescreen to their corresponding morph point in image 2. Each morph pointpixel of image 1 linearly taking on the positional, hue and brightnesscharacteristics of its corresponding morph point pixel in image 2. Andso on through the room of frozen butterflies.

Another method to deduce the 3-D shape of our subject would be toanalyze the silhouette of the subject as it appears from differentcamera locations. With several silhouettes compounded, a model of asimple subject would be formed mathematically. Known Blue Screen typeprocesses would be employed to select the subject from its background.More complex shapes would require a remote measurement method whichprobes each subject surface more accurately and thoroughly, such as theRoss type measurement system above, or perhaps a sophisticated radar orsonar measurement method. Methods of remote measurement have beendescribed which project a grid of lines onto the surface of an object.The distorted reflections of the grid lines are analyzed by computerprogram to determine the object's dimensions. Or, a raster, or rasters,of beams, or pulsed beams could be scanned across the subject area, fromseveral different angles, perhaps using different colored lasers. Acomputer program would then analyze the reflected shapes traced out asthese beams encounter a subject, and from these measurements deduce thesubject's geometry. The multiple rasters could scan, perhaps in multiplepasses, over the entire surface of our subject, leaving no surfaceunmeasured. An optically projected, un-scanned grid might containunilluminated surprises between grid lines.

Rather than interpolating to create more angular records of a subject anew method occurs to me as I type on May 13, 1994, of using transparentcameras (See FIG. 4) to form an array which would fill in the gaps inour traditional camera arrays. These cameras can see through oneanother, and we densely surround our subject with an array of beamsplitter camera devices, each of which would reflectively transfer onlysome of the light from the subject to a camera which corresponds to thatparticular beam splitter. Gaps in an initial array of such cameras wouldbe filled by a second array of such cameras, behind the first. Themembers of this second array would be slightly offset from the membersof the first, so that the second array captured angles of view betweenthe angles captured by the first array. And so on, with further arrays.If using identical cameras in both arrays, with all cameras in botharrays set to the same optical characteristics; then images capturedfarther away from the subject will be slightly different in size, andslightly different in perspective than images captured closely. Theimages dimensions from the various array ranks can later be conformedfor insertion into projectable series. If the distances from the subjectto the various array ranks are very nearly equal and/or the focal lengthof the cameras in each array is long, then image size and opticalperspective will change very little rank to rank.)

I add, for clarification, that the ROC target method of imagemanipulation would also be used in a silver halide camera array method.The ROC target images gathered by these cameras in array would bescanned into computer memory, where a computer program would analyze thetarget size and shape to determine array member orientation. Positionaland optical attributes of array members would be accordingly altered,manually or by remote control means, and/or subsequent images recordedby this camera array orientation would be altered, manually or bycomputer program, according to ROC target information, and printed inproperly registered series onto film or another display medium.

I add, for clarification, that in several embodiments, I envision arraymembers, mounted on moving platforms or suspended gondolas, so that theycan be moved about by remote control in a horizontal plane, rotatedhorizontally, raised and lowered in a vertical plane, and rotatedvertically. In other embodiments, array members will be more fixedlymounted, confined, say to an elliptical track about a subject, or morefixedly mounted so that the only degrees of movement would be horizontaland vertical rotation as in FIG. 1A, or locked down semi-permanently, orpermanently, to take in only one field of view.

It has been suggested that arcurate and circular arrays of cameradevices be arranged horizontally about a subject of interest, withoptical axes of said camera devices convergent upon a scene, whichcameras each record a different horizontally displaced image of thesubject, which displacement is employed to present sequences ofstero-optical image pairs to the viewer, which pairs are presented bycomplex means, such as specially built motion picture projectors,specially built and complex screens or other specially built and complexreflective or transmissive display devices, using specially built andcomplex obturating devices to separate and display left images toviewers' left eyes, and right images to viewers' right eyes. Theseeffects seem to be limited to the display of traditional motion pictureeffects, in 3D illusion, or the traditional display of still 3D images.

The foregoing description of the embodiments has been provided forpurposes of illustration and description. It is not intended to beexhaustive or to limit the disclosure. Individual elements or featuresof a particular embodiment are generally not limited to that particularembodiment, but, where applicable, are interchangeable and can be usedin a selected embodiment, even if not specifically shown or described.The same may also be varied in many ways. Such variations are not to beregarded as a departure from the disclosure, and all such modificationsare intended to be included within the scope of the disclosure.

Example embodiments are provided so that this disclosure will bethorough, and will fully convey the scope to those who are skilled inthe art. Numerous specific details are set forth such as examples ofspecific components, devices, and methods, to provide a thoroughunderstanding of embodiments of the present disclosure. It will beapparent to those skilled in the art that specific details need not beemployed, that example embodiments may be embodied in many differentforms and that neither should be construed to limit the scope of thedisclosure. In some example embodiments, well-known processes,well-known device structures, and well-known technologies are notdescribed in detail.

The terminology used herein is for the purpose of describing particularexample embodiments only and is not intended to be limiting. As usedherein, the singular forms “a,” “an,” and “the” may be intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. The terms “comprises,” “comprising,” “including,” and“having,” are inclusive and therefore specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof. The method steps, processes, and operations described hereinare not to be construed as necessarily requiring their performance inthe particular order discussed or illustrated, unless specificallyidentified as an order of performance. It is also to be understood thatadditional or alternative steps may be employed.

When an element or layer is referred to as being “on,” “engaged to,”“connected to,” or “coupled to” another element or layer, it may bedirectly on, engaged, connected or coupled to the other element orlayer, or intervening elements or layers may be present. In contrast,when an element is referred to as being “directly on,” “directly engagedto,” “directly connected to,” or “directly coupled to” another elementor layer, there may be no intervening elements or layers present. Otherwords used to describe the relationship between elements should beinterpreted in a like fashion (e.g., “between” versus “directlybetween,” “adjacent” versus “directly adjacent,” etc.). As used herein,the term “and/or” includes any and all combinations of one or more ofthe associated listed items.

Although the terms first, second, third, etc. may be used herein todescribe various elements, components, regions, layers and/or sections,these elements, components, regions, layers and/or sections should notbe limited by these terms. These terms may be only used to distinguishone element, component, region, layer or section from another region,layer or section. Terms such as “first,” “second,” and other numericalterms when used herein do not imply a sequence or order unless clearlyindicated by the context. Thus, a first element, component, region,layer or section discussed below could be termed a second element,component, region, layer or section without departing from the teachingsof the example embodiments.

Spatially relative terms, such as “inner,” “outer,” “beneath,” “below,”“lower,” “above,” “upper,” and the like, may be used herein for ease ofdescription to describe one element or feature's relationship to anotherelement(s) or feature(s) as illustrated in the figures. Spatiallyrelative terms may be intended to encompass different orientations ofthe device in use or operation in addition to the orientation depictedin the figures. For example, if the device in the figures is turnedover, elements described as “below” or “beneath” other elements orfeatures would then be oriented “above” the other elements or features.Thus, the example term “below” can encompass both an orientation ofabove and below. The device may be otherwise oriented (rotated 90degrees or at other orientations) and the spatially relative descriptorsused herein interpreted accordingly.

1. A method for producing a visual special effect comprising the stepsof: providing an array of cameras deployed along a curvilinear path witheach camera focused on a common scene containing a moving object;controlling time actuation of a series of proximate cameras along saidarray of cameras so that each camera begins to capture an image of thescene at a different point in space from each other; controlling a timelength of exposure of the proximate cameras to allow a controlled amountof light tunneling caused by said moving object to accumulate to providerespective light tunnel images; capturing said respective light tunnelimages on a plurality of cameras of the array of cameras, from themoving object to form a frozen image effect of the moving object; anddisplaying a sequence of respective light tunnel images from theplurality of cameras of the array of cameras in a motion picture mediumto create the special visual effect from a sequence of points of viewmoving along said curvilinear path while viewing said common scene,wherein the sequence points of view is rotated between first and secondangles on the motion picture medium to change the point of view of thefrozen image.
 2. The method for producing a visual special effectaccording to claim 1, wherein the frozen image effect comprises one ofslowing or stopping a moving object in time, in a series of images whileviewing the object from various angles along a sequentially changingperspective view along the path.
 3. The method for producing a visualspecial effect according to claim 1, wherein the frozen image effectcomprises stopping said moving object in time in a series of imageswhile viewing the object from various angles along a sequentiallychanging perspective view along the path.
 4. A method for producing avisual special effect comprising the steps of: providing an array ofcameras along a curvilinear path with each camera focused on a commonscene containing a moving object; controlling time actuation of a seriesof proximate cameras along said array of cameras so that each camerabegins to capture an image of the scene at a different point in spacefrom each other; controlling a time length of exposure of the proximatecameras to allow a controlled amount of light tunneling caused by saidmoving object to accumulate to provide respective light tunnel images;sequentially capturing said respective light tunnel images from themoving object on a plurality of cameras of the array of cameras; anddisplaying a sequence of respective light tunnel images from theplurality of cameras of the array of cameras in a motion picture mediumto create a special frozen image visual effect from a sequence of pointsof view which moves along said curvilinear path while viewing saidcommon scene; wherein the sequence of said respective light tunnelimages is rotated between first and second angles on the motion picturemedium to change the point of view of the moving object within thescene; and wherein the frozen image visual effect comprises at least oneof slowing and stopping movement of the moving object in the scene intime in a series of images while viewing the object from various anglesalong a sequentially changing perspective view along the path.
 5. Themethod according to claim 4, wherein the curvilinear shape is one ofcircular, arcuate, and dome-like.